Version: Mosquitto 2.9

Mosquitto PROXY protocol support

Introduction

Using Mosquitto as an entirely standalone broker works very well, but sometimes you may wish to place it behind a load balancer / proxy such as HAProxy. This provides certain advantages such as carrying out TLS termination in the proxy to reduce load on the broker.

It does have one disadvantage which is that only the proxy IP address is found in the broker logs - the client IP addresses are not seen.

Mosquitto Configuration

Since Mosquitto 2.9, this can be fixed using the PROXY protocol v2 support, which can be enabled using the enable_proxy_protocol_v2 option. This is the recommended mode when using HAProxy.

Example configuration of a port:

listener 1883
enable_proxy_protocol_v2 true
# Further listener settings

This page describes some different ways you can combine Mosquitto and HAProxy. It is not a complete guide to HAProxy.

Important: Enabling PROXY protocol support requires that the broker itself is not directly accessible on its network port. All communication must go through the proxy. If a client is able to connect to the broker directly, it is trivial to spoof connection information and this is especially important when using client certificates for mutual TLS on HAProxy. In that case, the contents of the PROXY header can directly indicate whether a client is allowed to connect so it must be protected.

It may be desirable to use a firewall to restrict access to the broker port.

HA Proxy Configuration examples

HA Proxy General setup

All examples presented will be of the haproxy.cfg file, typically located at /etc/haproxy/haproxy.cfg on a native Linux installation.

The first part of the config file contains the global and defaults sections which are going to be common to all the examples and not repeated.

Global section

This is a fairly standard global section, presented without comment.

global
        log /dev/log    local0
        log /dev/log    local1 notice
        user haproxy
        group haproxy
        daemon

        # Default SSL material locations
        ca-base /etc/ssl/certs
        crt-base /etc/ssl/private

        # See: https://ssl-config.mozilla.org/#server=haproxy&server-version=2.0.3&config=intermediate
        ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
        ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
        ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets

Defaults section

defaults
        mode    tcp
        timeout connect 5000
        timeout client  60000
        timeout server  60000

mode tcp - set TCP mode rather than HTTP mode by default. MQTT is not HTTP.
timeout connect 5000 - the timeout allowed for a client to connect, in milliseconds.
timeout client 60000 - if a client does not communicate in this interval, the connection will be closed by HAProxy.
timeout server 60000 - if the broker does not communicate in this interval, the connection will be closed by HAProxy.

The client and server timeout intervals should be chosen based on the intervals you expect to have communication occurring in your clients. If your communication is typically very sparse, the timeout should be chosen based on your keepalive interval, otherwise the quiet clients will be disconnected before they have chance to send a keepalive request.

Proxy configuration examples

Direct pass-through, without PROXY protocol

The most basic approach is to pass connections directly through HAProxy to Mosquitto without being modified. This means that if an unencrypted MQTT connection is made, it will pass through as an unencrypted connection, and if an encrypted MQTT connection is made it will pass through as an encrypted MQTT connection. Likewise, websockets connections are passed through unaffected.

Create a frontend, which is where HAProxy will listen for connections, and a backend, which is where the broker that HAProxy will connect to is defined.

frontend mqtt_frontend
         bind *:1883
         default_backend mqtt_backend

backend mqtt_backend
        server server1 127.0.0.1:1884 check on-marked-down shutdown-sessions

In this case, HAProxy is listening on all interfaces on port 1883 and will attempt to connect to the broker on address 127.0.0.1 on port 1884. In other words, HAProxy and Mosquitto are running on the same instance. This is not required.

The broker configuration could be with an encrypted or unencrypted listener:

Unencrypted:

listener 1884
# Further listener settings

Encrypted:

listener 1884
certfile <path/to/server.crt>
keyfile <path/to/server.key>
# Further listener settings

TLS termination with server certificate only, without PROXY protocol

Place your server certificate and private key in /etc/haproxy/certs/.

frontend mqtts_frontend
         bind 0.0.0.0:8883 ssl crt /etc/haproxy/certs/

backend mqtt_backend
        server server1 127.0.0.1:1883 check on-marked-down shutdown-sessions

The broker configuration should declare an unencrypted listener:

listener 1883
# Further listener settings

TLS termination with mutual TLS - client and server certificates, without PROXY protocol

In addition to the server certificate, you must also provide the CA certificate that will sign the client certificates, ask for verification of the client certificate and make the certificate required.

frontend mqtts_frontend
         bind *:8883 ssl crt /etc/haproxy/certs/ verify required ca-file /etc/haproxy/client-ca.crt
         default_backend mqtt_backend

backend mqtt_backend
        server server1 127.0.0.1:1883 check on-marked-down shutdown-sessions

The broker configuration should declare an unencrypted listener:

listener 1883
# Further listener settings

Direct pass-through, with PROXY protocol

To enable PROXY protocol v2 on HAProxy, add the send-proxy-v2 option to the backend.

frontend mqtt_frontend
         bind *:1883
         default_backend mqtt_backend

backend mqtt_backend
        server server1 127.0.0.1:1884 check on-marked-down shutdown-sessions send-proxy-v2

The broker configuration should declare an unencrypted listener and enable PROXY protocol v2 support. It is not possible to have a direct pass-through with encrypted connections on the broker.

listener 1883
enable_proxy_protocol_v2 true
# Further listener settings

TLS termination with server certificate only, with PROXY protocol

For TLS connections, use send-proxy-v2-ssl instead of send-proxy-v2. This ensures that TLS information is added to the PROXY header.

frontend mqtts_frontend
         bind *:8883 ssl crt /etc/haproxy/certs/
         default_backend mqtt_backend

backend mqtt_backend
        server server1 127.0.0.1:1883 check on-marked-down shutdown-sessions send-proxy-v2-ssl

On the broker side, use proxy_protocol_v2_require_tls true to ensure that only connections that were made using TLS are accepted on the broker. No other TLS configuration is required.

listener 1883
enable_proxy_protocol_v2 true
proxy_protocol_v2_require_tls true

TLS termination with mutual TLS - client and server certificate, with PROXY protocol

For TLS connections, use send-proxy-v2-ssl instead of send-proxy-v2. This ensures that TLS information is added to the PROXY header.

frontend mqtts_frontend
         bind *:8883 ssl crt /etc/haproxy/certs/ verify required ca-file /etc/haproxy/client-ca.crt
         default_backend mqtt_backend

backend mqtt_backend
        server server1 127.0.0.1:1883 check on-marked-down shutdown-sessions send-proxy-v2-ssl

The broker configuration uses require_certificate true to indicate that the broker should check the PROXY protocol header for the valid certificate result. No other TLS configuration is required.

listener 1883
enable_proxy_protocol_v2 true
proxy_protocol_v2_require_tls true
require_certificate true

TLS termination with mutual TLS - client and server certificate, with username and PROXY protocol

The Mosquitto option use_identity_as_username true can be used with the PROXY protocol support. This requires that the send-proxy-v2-ssl-cn option is used on HAProxy.

It is not possible to use use_subject_as_username with the PROXY protocol.

frontend mqtts_frontend
         bind *:8883 ssl crt /etc/haproxy/certs/ verify required ca-file /etc/haproxy/client-ca.crt
         default_backend mqtt_backend

backend mqtt_backend
        server server1 127.0.0.1:1883 check on-marked-down shutdown-sessions send-proxy-v2-ssl-cn

The broker configuration uses require_certificate and use_identity_as_username. No other TLS configuration is required.

listener 1883
enable_proxy_protocol_v2 true
proxy_protocol_v2_require_tls true
require_certificate true
use_identity_as_username true

Introduction​

Mosquitto Configuration​

HA Proxy Configuration examples​

HA Proxy General setup​

Global section​

Defaults section​

Proxy configuration examples​

Direct pass-through, without PROXY protocol​

TLS termination with server certificate only, without PROXY protocol​

TLS termination with mutual TLS - client and server certificates, without PROXY protocol​

Direct pass-through, with PROXY protocol​

TLS termination with server certificate only, with PROXY protocol​

TLS termination with mutual TLS - client and server certificate, with PROXY protocol​

TLS termination with mutual TLS - client and server certificate, with username and PROXY protocol​